Keyboard with input-sensitive display device

ABSTRACT

An input/output device is disclosed that includes an input-sensitive display screen and alphanumeric keys for entering characters. The input-sensitive display screen may be positioned proximate the alphanumeric keys and be capable of displaying graphical information and sensing user selection of the graphical information. The input-sensitive display screen may include a combination display/input region and a non-display input region. The touch sensitive display screen may be detachable from the alphanumeric region and may be operable in a detached configuration. The input/output device may usable with a method of input correction. The method may include displaying, on a secondary display, a first text unit, such as a character or a word, which corresponds with a second text unit shown on a primary display screen of a computing device. The method further involves receiving user selection of the first text unit and replacing the first text unit with a second text unit.

CROSS-REFERENCE To RELATED APPLICATIONS

This application is a continuation in part of co-pending U.S. patentapplication Ser. No. 10/427,430, filed Apr. 30, 2003.

BACKGROUND

As the field of computer science has evolved, a variety of data entrytechniques have been developed to enhance the individual's experienceand to make computers more versatile. For example, a typical computersystem, especially a computer system using graphical user interfaces foruser interaction may be optimized for accepting input from one or morediscrete input devices. Thus, an individual may enter text with akeyboard, and control the position of a pointer image on a displayscreen with a pointing device, such as a mouse, having one or morebuttons for activating selections associated with the location of thepointer. Some computing systems even have included a pen-like stylusthat can be used as a multipurpose data input device.

A variety of software applications have been developed that permit anindividual to form data files by entering characters with a keyboard orother input device. As utilized herein, the term character is intendedto encompass a symbol or other figure that may be entered by theindividual. Examples of characters include alphabetic characters,whether from the Roman, Cyrillic, Arabic, Hebrew, or Greek alphabets,for example. Furthermore, a character may be a numeral, a punctuationmark, or one of the various symbols that are commonly utilized inwritten text, such as $, #, %, &, or @, for example. In addition, acharacter may be one of the various symbols utilized in Asian languages,such as the Chinese, Japanese, and Korean languages. Groups of variouscharacters that form words or word-type units are hereby defined as atext unit.

Although conventional character entry with a keyboard is generallyconsidered to be a convenient and expedient process, an averageindividual frequently enters incorrect characters and is required tomodify the incorrect characters with the intended or correct characters.A common method of modifying the incorrect characters involves the useof a pointing device, for example a mouse or trackball. In order tomodify the incorrect characters, the individual will cease enteringcharacters and move one hand to the pointing device, and attempt tomanipulate the pointing device to position the cursor to the incorrectcharacter within the entire viewable X-Y field. The individual will thendelete the incorrect character, replace the incorrect character with theintended or correct character, and manipulate the pointing device tomove the cursor to another location which is typically the priorlocation of the cursor immediately before the edit. Alternately, variousspell-checking programs may be utilized, for example, and someindividuals may employ variations or combinations of these methods formodifying the incorrect characters. Such is normally done at thecompletion of the document in view of the potential disruption to thedata entering process. In either event, however, the individualgenerally redirects attention from the keyboard to the pointing devicewhen modifying the incorrect characters, which may decrease theefficiency of the individual, particularly when repetitively performed.Similar considerations apply to the correction of text units.

The error rate for an average individual utilizing a QWERTY keyboard forphonics-based Asian language input can be as high as 20% or worse. Theerror rate for an average individual utilizing a QWERTY keyboard toenter characters in the English language, for example, is generallysignificantly less. Accordingly, phonics-based Asian language input hasa greater error rate, which further decreases the efficiency of theindividual. The increased error rate for phonics-based Asian languageinput is directly related to the characteristics of the Asian languages.With regard to the Chinese language, for example, there are tens ofthousands of characters, but only approximately 400 correspondingpronunciations, and adding four tones to the pronunciations expands thetotal number of pronunciations to approximately 1600. Given therelatively large number of characters utilized in the Chinese language,many different characters have similar pronunciations and are,therefore, phonetically-similar. Coupled with the expanding Chinesevocabulary, the number of similar pronunciations introduces anintrinsically-high error rate in phonics-based Chinese language input.Similar considerations apply to the Japanese and Korean languages.

Concepts related to phonics-based Asian language input and aconventional method of correcting characters in phonics-based Asianlanguage input will now be discussed. For purposes of illustration, aChinese language version of phonics-based Asian language input, which isgenerally referred to as Pinyin, will be utilized herein. One skilled inthe relevant art will appreciate, however, that similar concepts may beapplied to other Asian languages. In general, a QWERTY keyboard isutilized for Pinyin input to enter Roman characters and combinations ofRoman characters that phonetically represent the intended Chinesecharacter. A software application then processes the Roman charactersand converts the Roman characters to a corresponding Chinese characterthat is similar-similar. As discussed above, many different charactershave similar pronunciations and are similar-similar. Accordingly, thesoftware application may convert the Roman characters to an incorrect orunintended Chinese character that is similar-similar to the intendedChinese character.

When an incorrect Chinese character is identified by the individual, inan existing system, the pointing device may be moved to place the cursorbefore and immediately adjacent to the incorrect Chinese character. Alist of potential replacement Chinese characters is then displayedadjacent to the incorrect Chinese character. The individual manipulatesthe pointing device to select the correct Chinese character from thelist. In many instances, the list may display only a portion of thetotal number of potential replacement Chinese characters, which maycorrespond with the Chinese characters that are most likely to be usedon a statistical basis. Accordingly, the individual may be required toscroll through numerous lists before finding the correct Chinesecharacter. Once the correct Chinese character is located and selected,the software application replaces the incorrect Chinese character withthe correct Chinese character and the individual may continue enteringcharacters until another incorrect Chinese character is identified. Thecursor remains at the location of the corrected Chinese character andthe individual will typically move the pointing device to the end of theline or document to continue entering data into the document.

Due to the relatively high error rate for phonics-based Asian languageinput, individuals are required to frequently identify incorrectcharacters and then identify the correct characters from lists ofpossible replacement characters, as discussed above. One skilled in therelevant art will recognize that this process may be time-consuming andinefficient given an approximate error rate of 20% or more for anaverage individual utilizing a QWERTY keyboard for phonics-based Asianlanguage input.

SUMMARY

Aspects of the present invention involve an integrated computerinput/output device having an input-sensitive display screen, such as atouch-sensitive screen or a proximity sensitive screen, and alphanumerickeys for entering characters. The input-sensitive display screen ispositioned proximate the alphanumeric keys and is capable of displayinggraphical information and sensing user selection of the graphicalinformation. The input-sensitive display screen may include acombination display/input region and a non-display input region, such asa touch pad region. The input-sensitive display screen may be detachablefrom the alphanumeric region and may be operable in a detachedconfiguration.

Other aspects of the invention involve a method for input correction.The method includes displaying, on a secondary display, a first textunit, such as a character or a word, which corresponds with a secondtext unit shown on a primary display screen of a computing device. Themethod further involves receiving user selection of the first text unitand replacing the second text unit with the first text unit. Thesecondary display may include a touch sensitive display screen and thestep of receiving the user selection of the first text unit may includesensing the user touching the touch sensitive display screen.

The advantages and features of novelty characterizing the presentinvention are pointed out with particularity in the appended claims. Togain an improved understanding of the advantages and features ofnovelty, however, reference may be made to the following descriptivematter and accompanying drawings that describe and illustrate variousembodiments and concepts related to the invention.

SUMMARY DESCRIPTION OF THE DRAWINGS

The foregoing Summary and following Detailed Description of the Drawingswill be better understood when read in conjunction with the accompanyingdrawings.

FIG. 1 is a schematic view of an operating environment.

FIG. 2 is an exemplary perspective view of the operating environment,which depicts a host computer, an output device, and an input device.

FIG. 3 is a perspective view of the input device.

FIG. 4 is a top plan view of the input device.

FIG. 5 is a top plan view of a portion of the input device.

FIG. 6 is a first configuration of a first graphical user interface thatmay be displayed on the output device.

FIG. 7 is a second configuration of the first graphical user interface.

FIG. 8 is a third configuration of the first graphical user interface.

FIG. 9 is a fourth configuration of the first graphical user interface.

FIG. 10 is a top plan view of another input device.

FIG. 11 is a top plan view of a portion of yet another input device.

FIG. 12 is a top plan view of a portion of a further input device.

FIG. 13 is a first configuration of a second graphical user interfacethat may be displayed on the output device.

FIG. 14 is a second configuration of the second graphical userinterface.

FIG. 15 is a third configuration of the second graphical user interface.

FIG. 16 is a fourth configuration of the second graphical userinterface.

FIG. 17 is a fifth configuration of the second graphical user interface.

FIG. 18 is a sixth configuration of the second graphical user interface.

FIG. 19 is a flow diagram illustrating one process of text unitreplacement.

FIG. 20 is a top plan view of a sensor.

FIG. 21A shows another configuration of a graphical user interfaceaccording to aspects of the invention.

FIG. 21B shows a configuration of an integrated input/output device thatillustrates aspects of the present invention.

FIG. 22 shows another configuration of an integrated input/output devicethat illustrates further aspects of the present invention.

FIG. 23 shows a further configuration of an integrated input/outputdevice that illustrates further aspects of the present invention.

FIG. 24 shows yet another configuration of an integrated input/outputdevice that illustrates further aspects of the present invention.

FIG. 25A shows a further configuration of a graphical user interfaceaccording to aspects of the invention.

FIG. 25B shows an additional configuration of an integrated input/outputdevice that illustrates aspects of the present invention.

FIG. 26 shows a further configuration of an integrated input/outputdevice that illustrates further aspects of the present invention.

FIG. 27 shows yet another configuration of an integrated input/outputdevice that illustrates further aspects of the present invention.

FIG. 28 shows functional components of an example configuration of aninput-sensitive display device for use with illustrating aspects of thepresent invention.

DETAILED DESCRIPTION OF THE DRAWINGS

The following discussion and accompanying figures disclose an inputdevice in accordance with aspects of the present invention, which mayinclude a combination input/output device. The input device may beutilized for entering characters or for other functionality. If anincorrect text unit, such as a character or a word, is inadvertentlyentered or is otherwise present, the input device can be utilized tomodify the incorrect text unit with the intended or correct text unit.More specifically, a sensor coupled to the input device may be utilizedby an individual to select the incorrect text unit and then replace theincorrect text unit with the correct text unit. The input device asdisclosed in many of the following configurations is a keyboard. Oneskilled in the relevant art will recognize, however, that a plurality ofother input devices having a sensor for replacing an incorrect text unitwill also fall within the scope of the present invention.

As used herein, the phrase “text unit” is defined as characters andgroups of characters that form words or word-type units. For example, inthe English language, the term text unit refers to characters and words.Accordingly, a character-based editing system can be used to replacebits of texts on a character-by-character basis, and a word-basedediting system can be used to replace text on a word-by-word basis. Atext unit-based editing system would be a generic term encompassing bothcharacter-based and word-based editing systems.

As will be appreciated by those of ordinary skill in the art, the inputdevice may be utilized in connection with a computer operatingenvironment. That is, the signals transmitted by the input device may begoverned by computer-executable instructions, such as program modules,executed by one or more computing devices. It may be helpful, therefore,to briefly discuss the components and operation of a typical operatingenvironment on which various embodiments of the invention may beemployed. FIGS. 1 and 2 illustrate examples of an operating environment10, in which various embodiments of the invention may be implemented.The operating environment 10 is only one example of a suitable operatingenvironment, however, and is not intended to suggest any limitation asto the scope of use or functionality of the invention. Other well knownoperating environments or configurations that may be suitable for usewith the invention include, but are not limited to server computers,hand-held or laptop devices, multiprocessor systems,microprocessor-based systems, programmable consumer electronics, networkPCs, minicomputers, mainframe computers, distributed computingenvironments that include any of the above systems or devices, and thelike.

The operating environment 10 includes a host computer 20 that isoperatively connected to an output device 30, such as a computermonitor, and an input device 40, which may be a keyboard, for example.One skilled in the relevant art will recognize that one or more datafiles may be processed by the host computer 20 and a signal may betransmitted to the output device 30, thereby directing the output device30 to render an image 32 on a display screen 31. The input device 40 isutilized in connection with the host computer 20 to enter data, whichmay take the form of a plurality of characters, as described in greaterdetail below.

The host computer 20 typically includes at least some form of computerreadable media. By way of example, and not limitation, computer readablemedia may comprise computer storage media and communication media.Computer storage media includes volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer readable instructions, data structures,program modules or other data. Computer storage media includes, but isnot limited to, RAM, ROM, EEPROM, flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, punched media, holographic storage, orany other medium which can be used to store the desired information.

In its most basic configuration, the host computer 20 typically includesa processing unit and system memory. Depending on the exactconfiguration and type of the host computer 20, the system memory mayinclude volatile memory (such as RAM), non-volatile memory (such as ROM,flash memory, etc.), or some combination of the two memory types.Additionally, the host computer 20 may also have mass storage devices,such as a removable storage device, a non-removable storage device, orsome combination of two storage device types. The mass storage devicescan be any device that can retrieve stored information, such as magneticor optical disks or tape, punched media, or holographic storage. As willbe appreciated by those of ordinary skill in the art, the system memoryand mass storage devices are examples of computer storage media.

The operating environment 10 may also have one or more additional inputdevices that are operatively connected to the host computer 20, such asa pointing device, microphone, or scanner, for receiving input from anindividual. Furthermore the operating environment 10 may have one ormore additional output devices operatively connected to the hostcomputer 20, such as a speaker, printer, or a tactile feedback device,for outputting data to an individual. Other components of the operatingenvironment 10 may include communication connections to other devices,computers, networks, servers, etc. using either wired or wireless media.All of these devices and connections are well know in the art and thuswill not be discussed at length here.

The output device 30 is depicted as a computer monitor that isoperatively connected to host computer 20. The image 32 may represent adata file, such as a text document, digital photograph, spreadsheet, orWeb page, for example. The input device 40 is depicted individually inFIGS. 3 and 4 as a keyboard. As is well-known in the art, one purpose ofthe input device 40 is to selectively enter data, which generally takesthe form of a plurality of characters and text units that may bedisplayed as a portion of the image 32 on the display screen 31. Asdescribed in the Background of the Invention section above, the termcharacter is intended to encompass a symbol or other figure that may beentered by the individual. Examples of characters include alphabeticcharacters, whether from the Roman, Cyrillic, Arabic, Hebrew, or Greekalphabets, for example. Furthermore, a character may be a numeral, apunctuation mark, or one of the various symbols that are commonlyutilized in written text, such as $, #, %, &, or @, for example. Inaddition, a character may be one of the various symbols utilized inAsian languages, such as the Chinese, Japanese, and Korean languages.Groups of various characters that are intended to form words orword-type units are included within the definition of a text unit.

A housing 41 forms an exterior of the input device 40, and a cord 42extends from the housing 41 in order to transmit signals from the inputdevice 40 to the host computer 20. Alternately, a conventional wirelessconnection between the input device 40 and the host computer 20 may alsobe utilized, as is well-known in the art. The signals that aretransmitted by the cord 42 are typically generated in response tomanipulation of one of a plurality of keys 43, buttons 44, or otherinput elements. Furthermore, the signals may be generated by a sensor 50that has an elongate configuration and is laterally-oriented withrespect to the input device 40.

The keys 43 of the input device 40 are generally arranged in aconventional manner and are positioned within an alphanumeric region 45,an arrow region 46, and a numberpad region 47, for example. Thealphanumeric region 45 has the configuration of a conventional QWERTYkeyboard, but may alternatively have the configuration of an AZERTYkeyboard, or any other desirable keyboard configuration within the scopeof the present invention. Accordingly, the alphanumeric region 45includes a plurality of keys 43 that are associated with individualalphabetic characters, which may be alphabetic characters of the Romanalphabet, for example. A first row of the alphanumeric region 45 mayalso include various keys 43 that are associated with numbers as theprimary characters. The keys of first row may also provide varioussymbols, such as !, @, #, $ and %, that are accessed in an alternativemode by depressing one a shift key simultaneously with such keys 43. Inaddition, the alphanumeric region 45 may include an enter key, a controlkey, and a space bar, for example, as is commonly known. Within thescope of the present invention, however, alphanumeric region 45 may havea plurality of configurations and may contain keys 43 with a variety offunctions or purposes.

Arrow region 46 includes four keys 43 u, 43 r, 43 d, and 431 that arerespectively associated with an up arrow, a right arrow, a down arrow,and a left arrow and are commonly utilized to move the cursor 33relative to display screen 31. Numberpad region 47 also includes aplurality of keys 43 with various numbers as the primary characters. Inoperation, the individual may utilize numberpad region 47 to quicklyenter numeric characters due to the arrangement of the keys 43 in thenumberpad region 47. The numberpad region 47 may also include a secondenter key. Additionally, the buttons 44 are positioned laterally acrossa top portion of input device 40. Suitable uses for the buttons 44include launching pre-designated software applications; adjusting thevolume or intensity of an output device, such as a speaker; modifyingpower levels of host computer 20; or providing basic controls for amedia player, for example. Additionally, an input region 48 may beprovided and laterally-spaced from the alphanumeric region 43 to provideenhanced input capabilities and may include a scroll wheel, anapplication switching device, editing keys (such as cut, copy, andpaste), and Internet browser control keys (such as forward and back).Details of aspects of the keyboard are disclosed in U.S. PatentApplication Number 20020159809, published Oct. 31, 2002, which is herebyincorporated by reference for its entirety. Based upon the abovediscussion regarding the layout and positioning of the keys 43 and theregions 45-48, one skilled in the relevant art will recognize that theinput device 40 has a generally conventional configuration, with theexception of the presence of the sensor 50 and related supporting keysas utilized in certain illustrative embodiments of the invention. Withinthe scope of the present invention, however, the various components ofthe input device 40 may have a plurality of alternate arrangements.

For reference purposes, the input device 40 has a back edge 11 distalfrom the individual during normal use, and a front edge 12 adjacent theindividual during normal use. Accordingly, an object is said herein tobe “behind” another object when it is between that object and the backedge 11. An object is said herein to be “directly behind” another objectwhen it is between that object and the back edge 11 and at leastpartially located within the lateral bounds of that object extending inthe front-to-back direction. An object is said herein to be “entirelydirectly behind” another object when it is between that object and theback edge 11 and entirely located within the lateral bounds of thatobject extending in the front-to-back direction. An object is saidherein to be “in front of” another object when it is between that objectand the front edge 12. Further, the keyboard 40 also has right and leftedges 13 and 14, respectively. The direction “lateral” defines thegeneral directions from the left edge 14 to the right edge 13 and fromthe right edge 13 to the left edge 14.

In addition to the features of existing keyboards, the input device 40also includes the sensor 50, which, in the depicted embodiment, ispositioned behind and/or adjacent to, and more specifically, directlybehind and entirely directly behind, the alphanumeric region 45. Asdepicted in FIGS. 3 and 4, the sensor 50 is adjacent to the rear portionof the alphanumeric region 45 and extends along and adjacent to thefirst row of the keys 43, which are generally associated with numericcharacters. Within the scope of the present invention, the sensor 50 maybe positioned in other portions of the input device 40. An advantage ofplacing the sensor 50 adjacent to the alphanumeric region 45 relates toaccessibility. The alphanumeric region 45 is a frequently utilizedportion of the input device 40. Accordingly, the hands of the individualare often positioned over the alphanumeric region 45 when utilizing theinput device 40. By placing the sensor 50 adjacent to the alphanumericregion 45, the sensor 50 is highly accessible to the individual and canbe manipulated by an individual with finger movements duringalphanumeric entry with little or no displacement of the wrist and arms.

The sensor 50 has an elongate configuration and is laterally-orientedwith respect to the input device 40. In other words, the length of thesensor 50 is greater than the width of the sensor 50, and the sensor 50extends generally from a right side of the input device 40 to a leftside of the input device 40. A ratio of the length to the width of thesensor 50 may be at least 3:1, but may also be 4:1 or 10:1 or greater,for example. In general, therefore, the ratio of the length to the widthof the sensor 50 is in a range of 3:1 to 10:1, but may be more dependingupon the specific application. Accordingly, the length of the sensor 50may be significantly greater than the width of the sensor 50.

Many conventional keyboards include a plurality of function keys thatare designated by F1, F2, F3, etc. The sensor 50 may be located in theposition that is conventionally reserved for the function keys, andactions associated with the function keys may be assigned to other onesof the keys 43, if desired. In applications where the function keys areto be preserved the sensor 50 may be positioned between the alphanumericregion 45 and the row of function keys such as shown in FIG. 10.

The sensor 50 may be a one-dimensional linear touch position sensor,such as a capacitive position sensing touchpad or other touch sensitivestrip. Touch-sensitive strips, pads and, other such devices arewell-known, such as the touch pad for cursor control commonly found onmany laptop computers. The present sensor 50 may take advantage of suchknown technology and/or be physically configured in any way to create atouch-sensitive device. The sensor 50 may be sensitive to human touchand/or to non-human touch such as from a pen or stylus-type pointer. Thesensor 50 also may be utilized without actually touching the surface ofthe sensor 50 with the pointer. For instance, the sensor 50 may beconfigured to detect the pointer position as it hovers just over thesurface of the sensor 50, as is known in the art.

The sensor 50 is coupled to the input device 40, and one or moreinterfaces and/or driving circuitry/software may be utilized to providecommunication between the sensor 50 and the host computer 20. Some orall of the interfaces and drivers, if any, may be located wherever isconvenient, such as all within the input device 40 all within the hostcomputer 20, or distributed between the two.

The sensor 50 may detect the position of a pointer along the sensor 50in any of a variety of ways. For example, the sensor 50 may be awell-known capacitance-type sensor that senses changes in capacitancebetween two or more conductive nodes within the sensor 50. Theconductive nodes do not electrically touch but generate a capacitance inthe dielectric junction between the nodes. When a pointer, such as ahuman finger or stylus, approaches the junction, the capacitance may bealtered due to the change in dielectric constant imposed by the pointer.Such a touch sensor 50 has the advantages of being able to sense notonly touch but also proximity and pressure. The conductive nodes may beof any shape and size, such as wires and/or plates. Depending upon thespecific embodiment, such a capacitive touch strip may detect pointerlocation with an accuracy of up to 0.001 inch or more. However, touchsensors with less precision can be effectively used.

The sensor 50 may alternatively be a resistance-type touch strip thatdetects variations in resistance based on the position of the pointeralong the touch strip. Or, the sensor 50 may be an inductance-type touchstrip that detects variations in inductance based on the position of thepointer along the touch strip. The sensor 50 may further incorporateinfrared-sensing, laser, and/or ultrasonic technology that senses theposition of the pointer.

Based upon the above discussion, the input device 40 may have thegeneral configuration of a conventional keyboard. In contrast with aconventional keyboard, however, the input device 40 includes the sensor50, which is positioned adjacent to the alphanumeric region 45. Withinthe scope of the present invention, the input device 40 may take theform of a plurality of other types of input devices, in addition to akeyboard. For example, the keypads of a conventional wireless telephoneare often utilized to input characters and form text units, and a sensorhaving the form of the sensor 50 may be utilized in conjunction with thekeypad. Furthermore, an input device having the configuration of apointing device, such as a mouse or trackball, may also be utilized insome situations to input characters, and a sensor having the form of thesensor 50 may be utilized in conjunction with the pointing device.Accordingly, the input device 40 may have the general configuration of avariety of input devices. For purposes of the following discussion,however, the input device 40 will be discussed with regard to akeyboard, as depicted in FIGS. 3-4.

The text unit system of the present invention includes the text unitediting sensor 50 and supporting input elements that assist in theselection of a text unit to be replaced, the confirmation of a text unitto be replaced, the presentation of alternative choices of replacementtext unit candidates for selection, the selection and/or confirmation ofa replacement text unit, and the exiting of the text unit replacementmode. These supporting input elements can take various types of inputelements and may include keys presently used on the keyboard of otheruses, such as the arrow keys 43 u, 43 d, 43 r, and/or 431, and/or mayinclude new or specifically dedicated input elements such as keys 51,52, and 53 as shown in FIGS. 3-5, key 51 and toggle 55 as shown in FIG.11, or toggle 56 as shown in FIG. 12, as is hereinafter described.

With reference to FIG. 5, a portion of the input device 40 that includesthe sensor 50 is depicted. A key 51 is positioned to the left of thesensor 50 and immediately adjacent to the sensor 50. Similarly, a key 52is positioned to the right of the sensor 50 and immediately adjacent tothe sensor 50, and a key 53 is positioned to the right of the key 52.For the embodiment of FIGS. 3-5, keys 51, 52, and 53 and the arrow keys431, 43 r, 43 u, and 43 d are supporting input elements that assist inthe selection of a text unit to be replaced, the confirmation of a textunit to be replaced, the presentation of alternative choices ofreplacement text unit candidates for selection, the selection and/orconfirmation of a replacement text unit, and the exiting of the textunit replacement mode.

In the configuration of FIG. 10, wherein the sensor 50 is positionedbetween the alphanumeric region 45 and the function keys, the arrow keys431, 43 r, 43 u, and 43 d are supporting input elements that assist inthe selection of a text unit to be replaced, the confirmation of a textunit to be replaced, the presentation of alternative choices ofreplacement text unit candidates for selection, the selection and/orconfirmation of a replacement text unit, and the exiting of the textunit replacement mode.

A third configuration is depicted in FIG. 11 and includes the sensor 50,the key 51, and a toggle key 55. The toggle key 55 is associated withtwo switches. When a right portion 55 r of the toggle key 55 isdepressed, a first of the switches is closed. Similarly, when a leftportion 551 of the toggle key 55 is depressed, a second of the switchesis closed. Accordingly, the individual may depress either the right orleft side of the toggle key 55 to effect an action. In thisconfiguration, the sensor 50 may be utilized as discussed above. In thisarrangement, the toggle key 55 and key 51 are supporting input elementsthat assist in the selection of a text unit to be replaced, theconfirmation of a text unit to be replaced, the presentation ofalternative choices of replacement text unit candidates for selection,the selection and/or confirmation of a replacement text unit, and theexiting of the text unit replacement mode.

A fourth configuration of the editing region of the input device 40 isdepicted in FIG. 12 and includes the sensor 50 and a three-way togglekey 56. The toggle key 56 includes three regions 57 a, 57 b, and 57 c,and the individual may depress one of the regions 57 a-57 c at a singletime. The sensor 50 may be utilized in conjunction with a plurality ofsupporting input elements, such as keys 51-53, toggle key 55, or togglekey 56. Thus, the regions 57 a, 57 b, and 57 c of toggle key 56 aresupporting input elements that assist in the selection of a text unit tobe replaced, the confirmation of a text unit to be replaced, thepresentation of alternative choices of replacement text unit candidatesfor selection, the selection and/or confirmation of a replacement textunit, and the exiting of the text unit replacement mode.

The manner in which the input device 40 is utilized to modify characterswill now be discussed. With reference to FIGS. 6-9, an example of agraphical user interface that may be displayed on the display screen 31as at least a portion of the image 32 is depicted. For purposes ofexample, the graphical user interface is depicted as a word processingsoftware application that is displayed within a window 33 having aborder 34 a, a taskbar 34 b, a toolbar 34 c, and a character processingarea 34 d. Such is conventional during the use of word processingprograms such as Microsoft WORD. The border 34 a defines the boundariesof the window 33 and may be adjustable in size to suit the preferencesof the individual. The taskbar 34 b is horizontally-oriented andpositioned adjacent to an upper portion of the border 34 a to provide aplurality of menus that permit the individual to perform various tasksrelated to the specific data file represented within the window 33.Furthermore, the toolbar 34 c is positioned directly below the taskbar33 b and provides a series of buttons that also permit the individual toperform various tasks related to the specific data file representedwithin the window 33.

The character processing area 34 d provides a space for the individualto enter various characters. More particularly, the character processingarea 34 d is a defined area where input from input device 40 is visuallyrendered. When the window 33 is initially rendered on display screen 31,the character processing area 34 d may be substantially blank, with theexception of a cursor 61, as depicted in FIG. 6, which shows theindividual the position where characters and text units will appear whenthe characters are input with the input device 40. The width of thecharacter processing area 34 d is herein defined as the distance on thedisplay covering the typeable area, e.g., from left margin to rightmargin. Assume for purposes of example that the individual wishes toinput the following series of text units: The quick fox jumps over thelazy brown dog. In order to accomplish this task, the individual merelydepresses a series of the letter keys 43 that correspond with thecharacters in the sentence, as is well-known in the art. On occasionhowever, the individual may inadvertently enter incorrect characters.Accordingly, the individual may inadvertently type the following seriesof characters: The quixk fox jumps over the lazy brown dog, as depictedin FIG. 7, wherein “quick” is inadvertently entered as “quixk”.

The sensor 50 may be utilized by the individual to correct theinadvertently-entered text unit. More specifically, the individual mayutilize the sensor 50 to modify a text unit 62 within the characterprocessing area 34 d. When entering the series of characters, theindividual intended to enter characters that form the word “quick”, butinadvertently-entered the text unit 62, which includes an “x” instead ofa “c”.

Conventional word processing software applications, such as MicrosoftWORD, provide a plurality of methods by which the individual may modifythe text unit 62 to the intended text unit. As is known in the art, theuser may utilize a pointing device to move the cursor 61 adjacent to thetext unit 62, highlight a portion of the text unit 62 with the pointingdevice, and then depress the key 43 that corresponds with the intendedcharacter. Alternately, an arrow key may be utilized to move the cursor61 to the text unit 62, and then a portion of the text unit 62 may bedeleted and replaced with the intended character. Furthermore, a spellchecking program may be utilized to modify the text unit 62. Onelimitation upon the spell checking program, however, is thatinadvertently-entered characters that form a recognized text unit maynot generally be recognized and modified with a spell checking program.

As an alternative to the conventional methods by which the individualmay replace the text unit 62 by using the text unit replacement systemof the present invention. The text unit system of the present inventionincludes the text unit editing sensor 50 and supporting input elementsthat assist in the selection of a text unit to be replaced, theconfirmation of a text unit to be replaced, the presentation ofalternative choices of replacement text unit candidates for selection,the selection and/or confirmation of a replacement text unit, and theexiting of the text unit replacement mode. These supporting inputelements can take various types of input elements and may include keyspresently used on the keyboard of other uses, such as arrow keys 43 u,43 d, 43 r, and/or 431, and/or may include new or specifically dedicatedinput elements such as keys 51, 52, and 53 as shown in FIGS. 3-5, key 51and toggle 55 as shown in FIG. 11, or toggle 56 as shown in FIG. 12.

Specifically, to utilize the text unit replacement system, a text unit,e.g., a word or character, to be replaced is selected. This is performedby making contact with the sensor 50. By contacting the sensor 50, thenormal data entry and editing routine within the word processing programis interrupted, and the text unit replacement mode is entered.

The location contacted on the sensor 50 relative to the side-to-sidelength of the sensor 50 will correspond to the location of the text unitthat is intended to be replaced. More specifically, the user willattempt to contact sensor 50 at a position corresponding to the locationof the text unit to be corrected relative to the width of the line oftext. For example, the text unit “quixk” in FIG. 7 is approximatelyone-quarter of the distance from a left side of the character processingarea 34 d to a right side of the character processing area 34 d.Accordingly, to select the word “quixk” to be replaced, the user maycontact an area on the sensor 50 that is approximately one-quarter ofthe distance from the left side of the sensor 50 to the right side ofthe sensor 50, which corresponds with a zone 54 that is depicted in FIG.5. Alternatively, the sensor 50 is mapped to correspond to the locationof the text unit to be corrected relative to the width of the specificline of text, i.e., from the beginning of the line to the end of theline.

Upon contacting the sensor 50, the initially selected text unit may begraphically accentuated by highlighting or otherwise graphicallydifferentiating the selected text unit from the other text units on theline. The action of making contact with the zone 54 will highlight oneor more text units that are approximately one-quarter of the distancefrom a left side of the character processing area 34 d to a right sideof the character processing area 34 d, as depicted in FIG. 8.

If the initially selected text unit to be replaced is not the desiredtext unit to be replaced, the user can modify the selection. In anexemplary embodiment, such is performed by the user moving the pointer,e.g., the user's finger, to the left or the right on the sensor, andalternative text unit can be designated to be replaced corresponding tothe final contact location on sensor 50. The line containing the textunit to be selected for replacement is designated for default to be theline where the cursor 61 is located. If desired, keys; not shown, may beprovided for input to move the line of text to be replaced up or down.

Accordingly, a proportional system is utilized to determine the specifictext unit that is selected. As discussed above, the text unit 62 wasapproximately one-quarter of the distance from a left side of thecharacter processing area 34 d to a right side of the characterprocessing area 34 d, and the individual contacted an area on the sensor50 that is approximately one-quarter of the distance from a left side ofthe sensor 50 to a right side of the sensor 50, which corresponds with azone 54. Accordingly, the specific text unit that is selected throughcontact with the sensor 50 may be selected based upon a proportionalitybetween the position of the text unit and the position of contact withthe sensor 50. The software application that controls the use of thesensor 50 may also be configured to only select a single text unit,i.e., either a single character or a single word. Accordingly, makingcontact with the sensor 50 may be utilized to select a single characterlocated on the line of the cursor 61.

Confirmation of the text unit to be replaced is then performed. In afirst embodiment, such confirmation is performed by the actuation of aninput element separate from sensor 50. For example, such confirmationcan be performed by pressing up arrow or down arrow 43 u or 43 l (FIG. 4and/or 10), or by pressing key 51 (FIG. 5 and/or 10), or by pressing theupper directional portion 57 c of toggle 56 (FIG. 12). In alternativeembodiments, confirmation may be performed by actuation of the sensor 50in a predetermined manner such as maintaining contact on the desiredlocation of the sensor 50 for a predetermined period of time or by asubsequent “tap” of the sensor 50 at the same location. Algorithms forcontrolling the interface regarding sensor 50 to determine when a touchis proper or inadvertent, when a predetermined contact has been met, orwhen a subsequent tap or double tap has occurred are known in the artand any suitable algorithm may be used.

Upon confirmation of a text unit to be replaced, alternative choices ofreplacement text unit candidates for selection are graphicallypresented. Preferably, and as shown in FIG. 8, the potential text unitcandidates for selection are presented in a horizontally orientationfacilitating the use of the sensor 50 for selection of a replacementtext unit. For example, as shown in FIG. 8, a list 63 that includespotential replacement text units, such as “quick”, “quirk”, “quark”, and“quack” is presented in a horizontal orientation immediately below theline of text containing the text unit to be replaced. The replacementcandidates can be provided based on a spelling mode wherein alternativeword options can be presented based on words having a similar spellingto the word to be replaced, and any desired spelling replacementtechnique can be used. Alternatively and/or in addition, the replacementcandidates can be provided based on a thesaurus mode wherein alternativeword options can be presented based on words having a similar meaning tothe word to be replaced, and any desired thesaurus replacement techniquecan be used.

In the event that the user is not satisfied with the replacementcandidates offered in the current horizontally oriented graphicaldisplay region, he or she may modify the list of candidate replacementtext units with the input of additional keys. By pressing apredetermined input element, such as arrow keys 431 and 43 r (FIG. 4),keys 52 and 53 (FIGS. 3-5), left and right regions 551 and 55 r oftoggle 55 (FIG. 11), or left and right regions 571 and 57 r of toggle 57(FIG. 12) and new set of replacement candidate text units is presented.Contacting the left and right keys/regions of these elements will togglemultiple sets of replacement candidate text units until the user locatesa desired text unit replacement candidate.

To select the desired text unit for replacement, the user identifies thedesired text unit for replacement (i.e., “quick”) from the list 63 ofpotential replacement text units (i.e., “quick”, “quirk”, “quark”, and“quack”). The location contacted on the sensor 50 relative to theside-to-side length of the sensor 50 will correspond to the location ofthe replacement text unit candidate that is intended to be used toreplace the previously selected text unit. More specifically, the userwill attempt to contact sensor 50 at a position corresponding to thelocation of the text unit to be corrected relative to the width of thegraphical display of potential text unit replacement candidates. Forexample, the candidate text unit “quick” in FIG. 8 covers approximatelythe first quarter of the distance from a left side of the candidate textunit replacement graphic to the right side of the candidate text unitreplacement graphic. Accordingly, to select the word “quick” to replace“quixk”, the user may contact an area on the sensor 50 that is withinthe zone spanning approximately 0% to 25% of the distance from the leftside of the sensor 50 to the right side of the sensor 50. Uponcontacting the sensor 50 in this region, the initially selectedreplacement text unit candidate may be graphically accentuated byhighlighting or otherwise graphically differentiating the selected textunit candidate from the other displayed text unit candidates within theset.

If the initially selected text unit candidate to be replaced is not thedesired replacement text unit candidate, the user can modify theselection. In an exemplary embodiment, such is performed by the usermoving the pointer, e.g., the user's finger, to the left or the right onthe sensor, and an alternative replacement text unit candidate (e.g.,“quirk”) can be designated corresponding to the final contact locationon sensor 50.

Confirmation of the replacement text unit candidate to be used is thenperformed. In a first embodiment, such confirmation is performed by theactuation of an input element separate from sensor 50. For example, suchconfirmation can be performed by pressing up arrow or down arrow 43 u or43 l (FIG. 4 and/or 10), or by pressing key 51 (FIG. 5 and/or 10), or bypressing the upper directional portion 57 c of toggle 56 (FIG. 12). Inalternative embodiments, confirmation may be performed by actuation ofthe sensor 50 in a predetermined manner such as maintaining contact onthe desired location of the sensor 50 for a predetermined period of timeor by a subsequent “tap” of the sensor 50 at the same location. Uponconfirmation of the desired replacement text unit to be used, theselected text unit to be replaced (e.g., “quixk”) is replaced with theselected replacement text unit candidate (e.g., “quick”) both at thedata level and graphically, and the result of such is shown in FIG. 9.Upon such confirmation, the text unit replacement mode is exited and theuser is then returned to the typical data entry and editing mode as isnormally used in the word processing program. Upon this return, thecursor 61 is located at its previous location (e.g., at the end of thestring of characters). In this location, the individual may merelycontinue entering characters when the editing process is complete. Anadvantage of this feature is that the editing process requires arelatively small degree of distraction when compared with conventionalediting processes.

Additionally, one of the supporting keys or an alternative key such asan escape (ESC) key may be used to exit the text unit replacement modeand return to the typical data entry and editing mode prior to thereplacement of the text unit if desired.

Based upon the above discussion, the sensor 50 may be utilized toconveniently modify incorrectly-entered characters or text units. Thealphanumeric region 45 is generally the most frequently utilized portionof the input device 40, and the hands of the individual are oftenpositioned over the alphanumeric region 45 when utilizing the inputdevice 40. By placing the sensor 50 adjacent to the alphanumeric region45, the sensor 50 is highly accessible to the individual. Accordingly,the individual may utilize the sensor 50 without having to significantlyadjust hand position with respect to the input device 40. For example,at least one hand is generally moved away from the input device 40 toutilize a pointing device to select and modify the text unit 62. Movinga hand away from the input device 40 generally requires that theconcentration of the individual change from the input device 40 to thepointing device, thereby interrupting thought processes that arecentered around the input device 40. The sensor 50, however, isincorporated into the input device 40 and the use of sensor 50 may beaccomplished with less disruption. Furthermore, the use of sensor 50 maybe a more expedient manner of modifying the text unit 62 than methodsinvolving the pointing device.

Input device 40 and the sensor 50 may also have application in thecontext of phonics-based Asian language input. Additionally, the textunit editing system may be used to replace characters as an alternativeto replacing words. Referring to FIGS. 13-18, an example of anothergraphical user interface that may be displayed on the display screen 31as at least a portion of the image 32 is depicted. For purposes ofexample, the graphical user interface is depicted as a softwareapplication for phonics-based Asian language input. More particularly,the graphical user interface is depicted as a software application forphonics-based Chinese language input that is displayed within a window33′ having a border 34 a′, a taskbar 34 b′, a toolbar 34 c′, and acharacter processing area 34 d′. The border 34 a′ defines the boundariesof the window 33′ and may be adjustable in size to suit the preferencesof the individual. The taskbar 34 b′ is horizontally-oriented andpositioned adjacent to an upper portion of the border 34 a′ to provide aplurality of menus that permit the individual to perform various tasksrelated to the specific data file represented within the window 33′.Furthermore, the toolbar 34 c′ is positioned directly below the taskbar34 b′ and provides a series of buttons that also permit the individualto perform various tasks related to the specific data file representedwithin the window 34′.

The character processing area 34 d′ provides a space for the individualto enter various characters. More particularly, the character processingarea 34 d′ is a defined area where input from input device 40 isvisually rendered. When the window 33′ is initially rendered on displayscreen 31, the character processing area 34 d′ may be substantiallyblank, with the exception of the cursor 61, as depicted in FIG. 13,which shows the individual the position where characters will appearwhen the characters are input with the input device 40.

In order to enter Chinese characters with phonics-based Asian languageinput, the individual enters Roman characters and combinations of Romancharacters that phonetically represent the intended Chinese character.Accordingly, Roman characters and combinations of Roman characters areentered and are converted to form Chinese characters. More specifically,the software application then processes the Roman characters andconverts the Roman characters to a corresponding Chinese character thatis phonetically-similar. Accordingly, the individual may enter variousRoman characters to display Chinese characters 64, as depicted in FIG.14. In order to display additional Chinese characters, the individualenters phonetically-similar Roman characters 65, as depicted in FIG. 15.As Roman characters 65 are entered, the software application provides alist 66 a having a plurality of potential Chinese characters 66 b thatmay be selected to replace the Roman characters 65. If the individualdoes not affirmatively select one of potential Chinese characters 66 b,the most probable candidate is automatically selected by the softwareapplication, as depicted in FIG. 16. Accordingly, the softwareapplication displays additional Chinese characters 64 following entry ofthe Roman characters 65.

Upon inspection of the Chinese characters 64, the individual maydetermine that an incorrect or undesired Chinese character 67, asdepicted in FIG. 16, was utilized. The user may then contact the sensor50 to enter the text unit replacement mode as described above. As thecontacted location of the sensor 50 corresponds to the location of theintended character to be replaced on its line of text, the user willattempt to contact the sensor 50 appropriately. For example, if thecharacter to be replaced is half way form the beginning of the line tothe end of the line of text, the user will want to contact the sensor inthe center (i.e., midway from the left of the sensor 50 to the right ofthe sensor). The position on the sensor 50 will designate the characterthat the user will want to replace and such will be graphicallyaccentuated. If the character initially selected for replacement is notthe intended character to be replaced, the user can modify his or herselection as described above, e.g., by moving the pointer along thesensor 50 to the proper location. Once the desired character to beselected has been made, the user can confirm the selection as describedabove, i.e., by using specific keys 431, 51, or 57 c, or by contactingthe sensor again in the same location.

Once the character to be replaced has been confirmed, a list 68 a ofpotential replacement Chinese characters 64 is graphically providedhorizontally and below the line of the selected character 67 to bereplaced. The list 68 a includes a set of candidate Chinese characters68 b for replacement. The depicted list 68 a includes nine alternateChinese characters 68 b. In many circumstances, the total number ofalternate Chinese characters 68 b may significantly exceed nine. Inorder to display the additional alternate Chinese characters 68 b, theuser may toggle through additional sets of candidates as describedabove, e.g., by actuating arrow keys 431 and 43 r (FIG. 4), keys 52 and53 (FIGS. 3-5), left and right regions 551 and 55 r of toggle 55 (FIG.11), or left and right regions 571 and 57 r of toggle 57 (FIG. 12)

To select one of the alternate Chinese characters 68 b, the usercontacts an area of the sensor 50 that correspond to a mapped region ofthe set of displayed candidates. If the initially selected charactercandidate to be replaced is not the desired replacement charactercandidate, the user can modify the selection. Such may be performed bythe user moving the pointer, e.g., the user's finger, to the left or theright on the sensor. Confirmation of the replacement text unit candidateto be used is then performed. Such confirmation may be performed asdescribed above, e.g., by actuating up arrow or down arrow 43 u or 43 l(FIG. 4 and/or 10), or key 51 (FIG. 5 and/or 10), or the upperdirectional portion 57 c of toggle 56 (FIG. 12). Alternatively,confirmation may be performed by actuation of the sensor 50 in apredetermined manner such as maintaining contact on the desired locationof the sensor 50 for a predetermined period of time or by a subsequent“tap” of the sensor 50 at the same location.

Upon confirmation of the desired replacement character 68 b to be used,the selected character 67 to be replaced is replaced with the selectedreplacement character candidate 68 b both at the data level andgraphically, and the result of such is shown in FIG. 18. Upon suchconfirmation, the text unit replacement mode is exited and the user isthen returned to the typical data entry and editing mode as is normallyused in the word processing program. Upon this return, the cursor 61 islocated at its previous location (e.g., at the end of the string ofcharacters). In this location, the individual may merely continueentering characters when the editing process is complete. An advantageof this feature is that the editing process requires a relatively smalldegree of distraction when compared with conventional editing processes.

Alternatively, in either the word-based or the character-based text unitreplacement system, the cursor 61 may be moved to (or immediatelyadjacent to) the text unit to aid in graphically providing visualfeedback of the text unit currently selected to be replaced. However,upon confirmation of the text unit candidate for replacement, or upon apremature exiting from the text unit replacement mode, the cursor 61 isreturned back to its prior position immediately before entering the textunit replacement mode which is normally at end of the characters toprepare for the entry of additional Roman characters, as depicted inFIG. 18.

As discussed in the Background of the Invention section, the error ratefor an average individual utilizing a QWERTY keyboard for phonics-basedAsian language input may be approximately 20% or more. Accordingly, theindividual may be required to edit one in five of the Chinese charactersthat are displayed. The sensor 50 provides a convenient andreadily-accessible device for efficiently selecting an incorrect Chinesecharacter and modifying the incorrect Chinese character to the intendedcharacter.

The above discussions regarding the editing of characters and text unitsprovided situations where the incorrectly-entered text unit is on thesame line as the cursor 61. When the individual contacts the sensor 50,text units located on the same line are selected. Utilizing the sensor50 to edit text units within a single line is one possible use for thesensor 50. In further embodiments of the invention, the sensor 50 may beutilized to select text units that are within a single sentence, even ifthe sentence is displayed on multiple lines. The sensor 50 may also beutilized to select text units within a single paragraph or within asingle document for example. Accordingly, the sensor 50 may be utilizedto edit characters and text units regardless of location on the displayscreen 31.

When the individual makes contact with the sensor 50, the relativeposition of the contact with respect to the length of the sensor 50determines the character or text unit that is selected for editing. Toassist the individual with determining the relative position of thefinger on the sensor 50, various queues may be added to the input device40. For example, the housing 41 may have markings corresponding with thevarious zones printed thereon. In addition, tactile queues, such asshallow protrusions, may be embossed or otherwise placed upon thesurface of the sensor 50. As the finger contacts the tactile queues, theindividual will gain feedback concerning the position of the finger withrespect to the sensor 50.

Referring to FIG. 19, a flow diagram illustrating the basic process forediting text units is disclosed. Initially, the operating environment 10is utilized in a normal mode to enter various characters and form textunits as is typically done in a word processing software application(step 80). Accordingly, the individual may depress combinations of thekeys 43 to display text units on the display screen 31, for example.While operating in the normal mode, the sensor 50 repeatedly detectswhether a pointer such as a finger of the user makes contact with thesensor 50 (step 81). If the individual does not make contact with thesensor 50, then the individual continues to operate in the normal mode.If, however, the individual does make contact with the sensor 50, thenthe operating environment 10 enters the text unit replacement mode (step82). In the text unit replacement mode, a text unit is selected andaccentuated based upon the position of contact with the sensor 50 (step83). If the position of sensor contact is modified (e.g., if theindividual slides the finger relative to the sensor 50), then adifferent text unit is accentuated based upon the new position ofcontact with the sensor 50 (step 84). If, however, the position ofsensor contact is not modified, as is determined by a confirmation ofthe selected text unit to be replaced, then various candidates forreplacing the accentuated text unit are listed and displayed (step 85).The various listed candidates for replacing the accentuated text unitmay reflect the most common replacement candidates. If the desired textunit does not appear within the candidate list, then the candidate listmay be revised to list additional candidates (step 86). If, however, thecandidate list need not be revised, the individual may select acandidate based upon a position at which contact is made with the sensor50 (step 87). If the position of sensor contact is modified (e.g., ifthe individual slides the finger relative to the sensor 50), then adifferent candidate is selected based upon the new position of contactwith the sensor 50 (step 88). If, however, the position of sensorcontact is not modified, as is determined by a confirmation of theselected text unit to be replaced, then the text unit is then replacedwith the selected text unit candidate (step 89). The applicationsoftware is then returned to the normal mode (step 90).

The text-based editing system discussed above permits the individual toedit text while drafting a document or otherwise entering text.Accordingly, the individual may proof-read the document while drafting,and make corrections before proceeding to create other portions of thedocument. The text-based editing system may also be utilized for aplurality of other applications. For example, documents that are scannedwith a scanning device are often converted into text documents. Althoughscan conversion programs are relatively accurate, the conversions mayintroduce unintended or incorrect text units that must be edited. Thetext-based editing system discussed above may be utilized, therefore, toedit scanned documents. Furthermore, the text-based editing system maybe utilized to edit documents created with voice-recognition software ordocuments that are translated. With regard to translation, thetranslation software may select a text unit that is a synonym of theintended text unit, but not the most proper text unit for the context.Accordingly, the text-based editing system may be utilized to replacetext units with synonyms.

In the above material, the sensor 50 is described as having an elongateconfiguration and being laterally-oriented with respect to the inputdevice 40. In one aspect of the invention, the sensor 50 may have alinear configuration. In a further aspect of the invention, however, thesensor 50 may be curved or may otherwise have a non-linearconfiguration, yet remain elongate and laterally-oriented. Referring toFIG. 20 a sensor 50′ is depicted. The sensor 50′ has a generally curvedconfiguration with a first end 51′ and a second end 52′. By makingcontact with the sensor 50′ adjacent to the first end 51′, text unitsgenerally positioned on the left side of the display screen 31 may beselected. Similarly, by making contact with the sensor 50′ adjacent tothe second end 52′, text units generally positioned on the right side ofthe display screen 31 may be selected. Furthermore, by making contactwith the sensor 50′ in a central area 53′, text units generallypositioned in a central area of the display screen 31 may be selected.Accordingly, the various sensor configurations within the scope of thepresent invention may exhibit a non-linear configuration.

Referring now to FIGS. 21A and 21B, an input device is shown in the formof input/output device 2140 for providing text-based editing accordingto further embodiments of the invention. Input/output device is asingle, integrated device that provides both input device functionalityand output device functionality. As such, input/output device 2140generally includes the same aspects and features as input device 40 withrespect to its input device capabilities for providing character inputsto host computer 20, except as pertaining to its sensor 2150. Asdiscussed further below, sensor 2150 includes an input-sensitive displaydevice 2150 that is configured to display text units or character unitsand to sense user selection of such units.

In addition to providing input functionality, device 2140 also acts as asupplemental output device along with primary monitor 30 shown in FIG.3. As is known in the art for dual monitor computer systems and asindicated in FIG. 1, host computer may be operatively connected to aplurality of output devices 30. For instance, host computer 20 mayinclude a second video display interface and driver (not shown) forproviding secondary display outputs to device 2140 shown in FIG. 21Balong with outputs to the primary monitor. However, rather than simplyproviding a secondary output device, input/output device 2140 shown inFIG. 21B provides the combined functionality of a keyboard type inputdevice and an input-sensitive display screen, such as a touch-sensitivescreen. Thus, input/output device 2140 provides character inputs viakeys 2143, supplemental display outputs via display device 2150, anddisplay inputs via user selection of data displayed on display device2150.

In general, display device 2150 is an input-sensitive display devicethat provides display functionality and permits various input modalsdepending upon the configuration, such as touch inputs, stylus inputs,gestures, handwriting inputs, 3-D gesture inputs, etc. For instance,input-sensitive display device 2150 may be a flat panel display formedfrom a liquid crystal display (LCD), an organic light emitting diodedisplay (OLED), a field emission display (FED), an electronic ink(e-ink) display, or a multi-touch sensing display surface providing 3-Dgesture inputs using vision technology, etc. Further, input-sensitivedisplay device 2150 may include a touch-sensitive display, aproximity-sensitive display, a resistance-sensitive display, adigitizer-sensitive display, or an optical-sensitive display, etc.

As illustrated in FIGS. 21A and 21B, input/output device 2140 permitsthe user to perform input correction via direct interaction withcandidate text units 2163 shown on display device 2150. The errorcorrection methods discussed above are generally applicable for use withinput/output device 2140, except that input/output device 2140 permitsthe user to directly select a desired text unit, such as by touching thecorrect text unit of text units 2163 shown on display device 2150. Assuch, error corrections methods are enhanced via direct user contactwith the correct text units, while the possibility of mistakes arereduced due to direct user selection of the desired text unit.

In the example word processing configuration illustrated in FIGS. 21Aand 21B, the operating environment 10 is utilized in a normal mode toenter various characters and to form text units shown in display 31 asis typically done via a word processing software application.Accordingly, the user may depress combinations of keys 2143 to displaytext units on the display screen 31, for example. In one embodiment, theoperating environment executing the word processing application maycontinually identify potential errors related to the current position ofthe cursor, for which it may continually display updated text unitcorrection options 2163 on the input-sensitive display device 2150. Inanother embodiment, while operating in the normal mode, theinput-sensitive display device 2150 may repeatedly detect whether apointer, such as a finger of the user, makes contact with theinput-sensitive display device. If the individual does not make contactwith the display device 2150, then the operating environment continuesto operate in the normal mode.

If, however, the individual does make contact with the input-sensitivedisplay device 2150, then the operating environment 10 enters a textunit replacement mode and provides text unit correction options 2163 onthe display device 2150. In the text unit replacement mode, a text unitis selected and accentuated based upon direct user selection of thedesired text unit. The various listed candidates 2163 for replacing theaccentuated text unit may reflect the most common replacementcandidates. If the desired text unit does not appear within thecandidate list, then the candidate list may be revised to listadditional candidates via user selection of next/back options 2151 ofthe display device. Once the user selects a replacement text unit, thenthe selected text unit replaces the selected text unit candidate and theapplication software is returned to its normal mode.

Referring now to FIG. 22, an input/output device 2240 is shown accordingto further embodiments of the invention. Device 2240 generally includesthe aspects and features of device 2140, except as pertaining touser-configurability of the display device 2250. As shown,input-sensitive display device 2250 may show virtual input options 2253,such as images of controls, buttons, soft keys, etc. In the exampleshown in FIG. 22, input options 2253 represent numeric keypad buttonsfound on conventional keyboards. As such, the user may select thevirtual keypad buttons shown on display device 2250 to perform therelated functionality, such as to enter the number corresponding to aselected keypad button. Numerous input configurations and controls,however, may be shown on display device 2250 as desired.

In one configuration, input/output device 2240 may be pre-programmed todisplay one of several pre-set input configurations from which the usermay select. In another embodiment, software (not shown) resident on thehost computer may be used to configure the virtual inputs shown ondisplay device 2250. In a further embodiment, input/output device 2240may include a processor 2255 and memory 2257 that cooperate to permitthe user to personally configure and/or program the functionality of thevirtual inputs options 2253 shown on the display device. As such,configuration instructions may be stored in memory 2257, which provideinstructions to processor 2255 to permit the user to interact withinput-sensitive display device 2250 and thereby configure the display toshow desired controls, buttons and other input options.

The instructions stored in memory 2257 may further permit the user tocreate specialized commands, routines, and macros, etc. that correspondto a control represented on the display. For example, the user maycreate a virtual hotkey button (not shown) to be shown on the display,which, when selected, causes a particular application to launch (e.g., abrowser application) or that generates a particular command (e.g.,places the computer system in a sleep mode).

As further shown in FIG. 22, display device 2250 may be divided intovarious operational regions 2259 and 2261. Region 2259 may be reservedas a combination display/input region for displaying and selectingcharacters (e.g., virtual input options 2253), whereas region 2261 maybe configured as a non-display input region to receive directionalinputs, such as mouse-type directional inputs, cursor control inputs,scrolling inputs, etc., which are conventionally provided via touchpads. Additionally, discrete input controls may be provided proximatethe input-sensitive display device 2250 to enhance user interactiontherewith. For instance, selection buttons 2263 may be provided, as wellas a rotary disk 2265, toggle switches 2267, and roller wheel 2269,which can provide specific functionality to the display device and/ormay be configured by the user as desired to provide custom functionalityto the display device. For instance, rotary disk 2265 may providedefault rotation functionality to images displayed on the displaydevice, or it may be custom configured by the user to switch betweenparticular displays. For instance, rotary disk 2265 may becustom-configured by the user to switch between various configurationsof virtual inputs 2253 like a scientific calculator arrangement and aconventional calculator arrangement. Similarly, roller wheel 2269 may bea conventional scroll wheel that provides movement input instructionsfor displays on display device 2250. Likewise, toggle switches 2267 mayprovide default functionality, such as switching between virtualdisplays on the display device, or it may be configurable for customfunctionality.

In an integrated keyboard/input-sensitive display device, such as device2240 shown in FIG. 22, a substantially full-sized keyboard 2245 iscoupled with the input-sensitive display device 2250. In such aconfiguration, a display screen 2247 of the input-sensitive display maybe a flat panel display screen having a width of about 4 inches to 10inches. Such a range of sizes permits the input/output display device2250 to have a flexible range of options and display configurations toeffectively assist the user with interacting with the computingenvironment. More particularly, display screen 2247 preferably has adisplay screen width of 7.8 to 8.5 inches, which generally matches thewidth of conventional paper, and more preferably has a display screenwidth of 8.0 to 8.3 inches to accommodate 8.5×11 inch paper and A4 sizepaper (210 mm width). Nonetheless, these sizes are not unduly large asto prevent the display screen from becoming overwhelmingly large withrespect to the substantially full-sized keyboard 2245.

Input/output device 2240 may include a single interface 2241 with thehost computer 20, such as a wired or a wireless connection. Forinstance, interface 2241 may include a universal serial bus (USB)interface, a FIREWIRE interface, a home audio video interoperability(HAVI) interface, or other types of interfaces. Further, interface 2241may include a wireless BLUETOOTH interface, a wireless personal areanetwork (WPAN) interface, a wireless local area network (WLAN)interface, a wireless fidelity network (WiFi) interface, a cellularinterface, an infrared interface, or other type of wireless interface.In addition, interface 2241 may include a plurality of interfaces, suchas a first interface for the keyboard 2245, an output interface from thehost computer to the display device 2250, and/or an input interface fortransmitting to the host computer user-selected display inputs fromdisplay device 2250.

FIG. 23 illustrates an example input/output device 2340 that generallyincludes the aspects and features of device 2240, except as pertainingto the detachability of the input-sensitive display device 2350. Asshown, input-sensitive display device 2350 includes a keyboard interface2361, such as a USB connection, for operatively connecting the devicewith keyboard 2345. The keyboard interface 2361 is a schematicrepresentation of an interface between keyboard 2245 and display device2250, which may include a first part of a USB connection 2391 and asecond part of a USB connection 2393. Such a connection permits thedisplay device 2350 to quickly connect with the keyboard and, thereby,to connect quickly with the host computer to which the keyboard isconnected. Alternatively, display device 2350 may independently connectwith the host computer and, thus, simply be placed proximate thekeyboard to facilitate their collective use. FIG. 23 also shows avirtual slider 2335 displayed on input-sensitive display device 2350,which can permit the user to scroll content shown on the display device.FIG. 23 further shows a virtual page turning control 2337, which canpermit the user to turn pages shown on the display device.

FIG. 24 illustrates another example input/output device 2440 thatgenerally includes the aspects and features of device 2340, except thatit includes a second keyboard interface 2363 and it provides additionalfunctionality. The second keyboard interface permits the input-sensitivedisplay device 2350 to be placed in a lateral configuration as shown inFIG. 23, or in a vertical configuration as shown in FIG. 24 according tothe user's preference. The second keyboard interface 2362 is representedschematically as an interface between the keyboard and the displaydevice, which may include a first part of USB connection (not shown) onthe keyboard and a second part on the display device, similar to parts2391 and 2393 of FIG. 23. Further, it is understood that either or bothof interfaces 2361 and 2363 may be wireless connections between thekeyboard and the display device, such as a BLUETOOTH connection. Inalternate configurations, the display device and the keyboard mayindependently communicate with host computer 20 to provide the describedfunctionality. The lateral configuration of FIG. 23 may be advantageousfor providing inputs to the display device, such as touch, stylus or 2Dgesturing inputs, whereas the vertical configuration may be advantageousfor permitting multi-monitor or mini-monitor uses of display device2350. For instance, the computing environment may be configured toprovide supplemental information to input-sensitive display device 2350for use as a secondary monitor or a mini-monitor. In one example, hostcomputer 20 may be configured to provide text messaging information ondisplay device 2350 rather than on the primary monitor 30. Of course, awide variety of data types and application examples may take advantageof a secondary, mini-monitor. In the vertical configuration shown,display device 2350 may include angular adjustments to orient thedisplay at a desired orientation.

The vertical configuration shown in FIG. 24 may also be beneficial forfacilitating multi-modal inputs to the input-sensitive display device2350, such as hand gestures, speech or vision inputs. For example,input-sensitive display device 2350 may include a multi-touch sensingdisplay surface, such as the multi-touch control surface provided by theJAZZ MUTANT company of Pessac, France, which can sense various 2D and 3Dgesture inputs. Further, input-sensitive display device 2350 may furtherinclude an image sensor or a video sensor 2365, a microphone 2367, and aspeaker 2369, which may further enhance user interaction with thedisplay device in the vertical configuration. In the example of FIG. 24,input-sensitive display device 2350 is providing video camerafunctionality and displaying a video image 2371 received by hostcomputer 20.

FIG. 25 shows a further example input/output device 2540 that generallyincludes the aspects and features of device 2440 in the verticalconfiguration. In addition, the example shown further illustrates use ofthe input/output device 2540 for performing error correction methods,such as the methods discussed along with FIGS. 21A and 21B, as well asFIGS. 6-9 and 13-19. As noted for FIGS. 21A and 21B, direct interactionwith text correction options 2563 can facilitate and simplify usercorrection of information in a document. The vertical configurationshown in FIG. 25 may be particularly advantageous for viewing andinteracting with large number of correction options that may be providedwhen a user interacts with Asian languages and various character optionsrelated thereto.

FIG. 26 shows a further example input/output device 2640 that generallyincludes the aspects and features of device 2340, as well as beingconfigured to interact with a stylus 2673. Via use of stylus 2673, auser can select, highlight, and/or write on the digitizinginput-sensitive display device 2650. Examples of suitable digitizingstylus devices 2673 include electromagnetic pen digitizers, such asMUTOH or WACOM pen digitizers. Other types of pen digitizers, e.g.,optical digitizers, may also be used. Input-sensitive display device2650 and/or host computer 20 operatively connected therewith interpretgestures made using stylus 2673 in order to manipulate data, enter text,create drawings, and/or execute conventional computer application taskssuch as spreadsheets, word processing programs, and the like.

The stylus 2673 may be equipped with one or more buttons or otherfeatures to augment its selection capabilities. In one embodiment, thestylus 2673 could be implemented as a “pencil” or “pen”, in which oneend constitutes a writing portion and the other end constitutes an“eraser” end, and which, when moved across the display, indicatesportions of the display are to be erased. Other types of input devices,such as a mouse, trackball, or the like could be used. Additionally, auser's own finger could be the stylus 2673 and used for selecting orindicating portions of the displayed image on input-sensitive displaydevice 2650.

In various embodiments, the computing system 10 provides an ink platformas a set of COM (component object model) services that an applicationcan use to capture, manipulate, and store ink. One service enables anapplication to read and write ink using the disclosed representations ofink. The ink platform may also include a mark-up language including alanguage like the extensible markup language (XML). Further, the systemmay use DCOM as another implementation. Yet further implementations maybe used including the Win32 programming model and the Net programmingmodel from Microsoft Corporation.

FIG. 27 shows a further example input/output device 2740 that generallyincludes the aspects and features of device 2340, as well as beingconfigured to provide menus 2781 and menu options 2783 on theinput-sensitive display device 2750 for an application being processedby computer 20 and shown on primary monitor 30. As illustrated,input-sensitive display device 2750 can improve user interaction withgraphical interface items, such as menus, by displaying such items onthe input-sensitive display device 2750. Such a display places the userinterface items in close proximity to the user's fingers while the useris entering data on keyboard 2745, which improves the user's ability toselect a menu item without needing to reach for a mouse or another inputdevice. In one configuration, the menus may be provided automaticallywhile the user is working with a particular application. In anotherconfiguration, the user may select a particular hot key on keyboard 2745to cause menu 2791 to be displayed. In a further configuration, the usermay interact with input-sensitive display device 2750 to request themenu display.

FIG. 28 shows functional components of an example configuration of aninput-sensitive display device 2850, which may be used withpreviously-discussed embodiments and configurations. In addition, asshown, display device 2850 may include functional components to permitit to operate separately from host computer and/or other input devices.However, it may also be configured only for operation with the hostcomputer and/or an input device. In the stand-alone configurations,display device 2850 may mate with a host computer, a keyboard, oranother device to provide the functionality discussed above, and mayprovide stand-alone functionality to operate as a personal digitalassistant, a mobile terminal device, a cellular telephone, and a remotecontrol device, etc.

As shown, the display device may include a processor 2860, a memory2862, a display 2828, and input mechanisms/keys 2832. The memory 2862generally includes both volatile memory (e.g., RAM) and non-volatilememory (e.g., ROM, PCMCIA cards, etc.). An operating system 2864 isresident in the memory 2862 and executes on the processor 2860, such asthe Windows® CE operating system from Microsoft Corporation or anotheroperating system.

One or more application programs 2866 are loaded into memory 2862 andrun on the operating system 2864. Examples of applications include emailprograms, scheduling programs, PIM (personal information management)programs, word processing programs, spreadsheet programs, Internetbrowser programs, and so forth, as well as applications specific to theembodiments and configurations discussed along with FIGS. 21-28, such asinput device configuration software. The input-sensitive display device2850 may also have a notification manager 2868 loaded in memory 2862,which executes on the processor 2860. The notification manager 2868handles notification requests from the applications 2866.

The input-sensitive display device 2850 may have a power supply 2870,which is implemented as one or more batteries. The power supply 2870might further include an external power source that overrides orrecharges the built-in batteries, such as an AC adapter or a powereddocking cradle.

The input-sensitive display device 2850 is also shown with an audiogenerator 2874, a microphone 2876, and an image capture system 2878.These devices are directly coupled to the power supply 2870 so that whenactivated, they remain on for a duration dictated by the notificationmechanism even though the processor and other components might shut downto conserve battery power.

The present invention is disclosed above and in the accompanyingdrawings with reference to a variety of embodiments. The purpose servedby the disclosure, however, is to provide an example of the variousfeatures and concepts related to the invention, not to limit the scopeof the invention. One skilled in the relevant art will recognize thatnumerous variations and modifications may be made to the embodimentsdescribed above without departing from the scope of the presentinvention, as defined by the appended claims. In particular, it isunderstood that various features described herein may be combined intovarious embodiments of the invention. For instance, features related tothe digital pen functionality described along with FIG. 26 may becombined with the menu functionality of FIG. 27, which may reside in thenon-separable configuration of input/output device 2240 shown in FIG.22. However, in order to avoid excessive drawings showing manycombinations of features for various embodiments, it is understood thatthe features and aspects described herein may be combined in variousarrangements. It is further understood that various embodiments may beformed from separate units, such as a keyboard and an input-sensitivedisplay device, which are combined to provide desired functionality. Assuch, the separate units may constitute a kit for providing anembodiment of the invention.

1. An integrated input/output device for a computer system having aprimary display, the integrated input/output device comprising: aplurality of alphanumeric keys for receiving character inputs from auser; an input-sensing display device configured to provide displayssupplemental to the primary display of the computer system and toreceive display selection inputs from the user, the input-sensingdisplay device positioned proximate the plurality of alphanumeric keys;and an interface configured to provide the character inputs and thedisplay selection inputs to the computer system and to receivesupplemental display data for displaying on the input-sensing displaydevice.
 2. The integrated input/output device recited in claim 1,wherein the input-sensing display includes one of a touch-sensitivedisplay device, a proximity-sensitive display device, aresistance-sensitive display device, a digitizer-sensing display device,and an optical-sensing display device.
 3. The integrated input/outputdevice recited in claim 1, further comprising a keyboard including thealphanumeric keys, wherein the input-sensing display device includes aflat panel display having a width of 4 inches to 10 inches.
 4. Theintegrated input/output device recited in claim 3, wherein the flatpanel display has a width of about 7.8 to 8.5 inches.
 5. The integratedinput/output device recited in claim 1, wherein the interface includes:a first interface for transmitting to the computer system characterinputs received via the user-selection of the alphanumeric keys; asecond interface for providing the display selection inputs to thecomputer system; and a third interface for receiving supplementaldisplay data for displaying on the input-sensing display device.
 6. Theintegrated input/output device recited in claim 1, wherein the interfaceincludes a wireless interface with the computer system.
 7. Theintegrated input/output device recited in claim 6, wherein the wirelessinterface includes one of a BLUETOOTH interface, a wireless personalarea network (WPAN) interface, a wireless local area network (WLAN)interface, a wireless fidelity network (WiFi) interface, a cellularinterface, and an infrared interface.
 8. The integrated input/outputdevice recited in claim 1, wherein the interface includes a wiredinterface with the computer system including one of a universal serialbus (USB) interface, a FIREWIRE interface, and a home audio videointeroperability (HAVI) interface.
 9. The integrated input/output devicerecited in claim 1, further comprising a first structure housing thealphanumeric keys and a second structure housing the input-sensingdisplay device, wherein the interface includes a first interface part onthe first structure and a second interface part on the second structure,the first and second interface parts configured to provide anoperational connection when mated together.
 10. The integratedinput/output device recited in claim 1, wherein the input-sensingdisplay device includes a directional input region for receivingdirectional inputs from a user and a character display/selection regionfor displaying characters and receiving user selection of thecharacters.
 11. The integrated input/output device recited in claim 1,wherein the input/output device is configured to provide auser-configurable display of virtual input keys.
 12. The integratedinput/output device recited in claim 1, further comprising: a memory;and a processor for performing steps comprising: receiving userinstructions for configuring a display of virtual input keys; displayingthe user-configured virtual input keys; receiving user selection of oneof the user-configured input keys; translating the user selection intoone of a command and a character input for the computer system; andproviding the one of a command and a character input to the computerdevice.
 13. The integrated input/output device recited in claim 1,wherein the input-sensing display device is removable from the pluralityof alphanumeric keys.
 14. The integrated input/output device recited inclaim 13, further comprising a display device interface operativelyconnecting the alphanumeric keys and the input-sensing display devicewhen connected with each other.
 15. The integrated input/output devicerecited in claim 13, wherein the input-sensing display device comprisesa computing device having functionality independent of the integratedinput/output device.
 16. The integrated input/output device recited inclaim 15, wherein the input-sensing display comprises a personal digitalassistant (PDA).
 17. An input-sensing display device comprising: aninput-sensing display; a communications interface; and a processorcoupled to the input-sensing display and the communications interfacecoupled to provide independent computing functionality while thecomputing device is in an independent mode of operation and, while inthe input/output device mode of operation, provide displays supplementalto a primary display of the host computer system and receive displayinputs from a user for the host computer system based on thesupplemental displays.
 18. The computing device recited in claim 17,wherein the computing device provides personal digital assistant (PDA)functionality while in the independent mode.
 19. A method of interactingwith a computer system having a primary display, the method comprising:displaying user-selectable inputs on a supplemental display locatedproximate a plurality of alphanumeric keys; receiving user-selection ofone of the user-selectable inputs; and identifying the user-selection tothe computer system.
 20. The method of claim 19, wherein the step ofreceiving user-selection includes sensing a user's touch of the one ofthe user-selectable inputs.